Inverse Reinforcement Learning with Explicit Policy Estimates

Various methods for solving the inverse reinforcement learning (IRL) problem have been developed independently in machine and economics. In particular, method of Maximum Causal Entropy IRL is based on perspective entropy maximization, while related advances field economics instead assume existence unobserved action shocks to explain expert behavior (Nested Fixed Point Algorithm, Conditional Choice Probability method, Nested Pseudo-Likelihood Algorithm). this work, we make previously unknown connections between these from both fields. We achieve by showing that they all belong a class optimization problems, characterized common form objective, associated policy objective gradient. demonstrate key computational algorithmic differences which arise due an approximation optimal soft value function, describe how leads more efficient algorithms. Using insights emerge our study identify various scenarios investigate each method's suitability problems.